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ABSTRACT 


A Cost and Operational Effectiveness Analysis (COEA) is performed for three alternative anti- 
surface warfare (ASUW) platforms that will conduct operations in multi-service, regional scenarios. 
Estimated program costs, historical cost variances, and measures of operational effectiveness are 
determined for each COEA alternative, and service life extension effects are examined. The data is 
merged in a mixed-integer optimization model, MPAMOD1, that develops the best implementation 
plan for each alternative. The solution of choice is an ASUW Improvement Program modified P-3C 
whose service life is extended through a Sustained Readiness Program. Historical cost variance of 
P-3C cost estimates proves inconsequential over the planning horizon. A second question is then 
examined, that of the cost effectiveness of major modification programs versus new production 
aircraft. Cost effectiveness of major modification programs becomes doubtful only when modification 


costs exceed 150% of original cost estimates. 
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I. INTRODUCTION 


The Department of Defense (DOD) approaches the 21st 
century facing critical decisions involving the composition of 
future armed forces. DOD managers must shape, from a group of 
disparate options, a force structure that balances combat 
effectiveness and cost while still meeting mission needs. This 
thesis examines part of one resource allocation decision 
involving alternative anti-surface warfare (ASUW) platforms. 
These platforms are required for use in multi-service, limited 
area, shallow water operations. 

The DOD Cost and Operational Effectiveness Analysis (COEA) 
process is used to examine several variants of three 
alternative ASUW platforms currently being considered: 
existing, modified, and new production P-3 maritime patrol 
aircraft. This thesis first develops detailed cost estimates, 
which incorporate historical cost variance calculations. The 
cost estimates are merged with measures of operational 
effectiveness (MOE) in an optimization model. The model 
maximizes P-3 fleet effectiveness subject to budget 
limitations and annual inventory, MOE, and average fleet age 
goals. It then provides a schedule for modification 
implementation or new aircraft procurement. 


After the best COEA alternative is determined, a new 


question is examined. The model is used to compare the cost 














effectiveness of modified versus new production P-3 aircraft. 
In particular, the effects of different service lives and 


procurement costs on life cycle costs are evaluated. 


A. AIRCRAFT PROCUREMENT AND FISCAL REALITY 

By 1997, the defense budget, as proposed, will experience 
a 41% real decline compared to the peak year of the Reagan 
defense buildup, fiscal year 1985 (FY 1985) [{Ref. 1]. As world 
tensions between major powers ease and emphasis is placed on 
solving domestic problems, actual FY 1997 defense spending 
could fall to much lower levels [Ref. 2]. 

Weapon systems procurement comprises a sizable portion of 
the defense budget. As expected, the DOD procurement budget 
has also declined precipitously since 1985, from a high of 
$115 billion in constant 1985 dollars to approximately $60 
billion in 1992 [Ref. 3]. The procurement budget cuts caused 
termination of ongoing acquisition programs, such as the 
Army’s Apache helicopter and the Air Force’s F-15E aircraft, 
before follow-on systems were evailable. Other programs 
experiencing acquisition problems, such as the Navy’s A-12 and 
P-7 aircraft programs, have been cancelled outright before 
production. 

The DOD procurement budget cuts are occurring when Naval 
aircraft procurement needs are increasing due to aircraft 
obsolescence and aging. According to the Congressional Budget 


Office (CBO) [Ref. 4], the Aircraft Procurement Navy (APN) 











accounts have experienced an average real growth of 7% 
annually from 1987 through 1992. They will stabilize in 1994 
at approximatesy 9.5 percent of total DON budget authority 
(Ref. 5], yielding approximately $6 Dillion for aircraft 
procurement and modification. Shortfalls of 176 fighter and 
attack aircraft are projected during this same time period, 
based on aircraft requirements for 15 aircraft carriers. Even 
with reduced aircraft needs given fewer aircraft carriers (a 
reduction to 12 by FY 1994 is proposed [Ref. 1]), effective 
use of APN dollars is paramount as Naval Aviation struggles 
with the conflicting requirements of aircraft type, quantity, 
effectiveness, cost, and age. 

As budget resources become scarcer, effective allocation 
tools gain in importance. The COEA is one tool that encourages 
the effective resource allocation needed to deal with these 


requirements. 


B. THE COST AND OPERATIONAL EFFECTIVENESS ANALYSIS 
The use of the COEA is endorsed by the Office of the 
Assistant Secretary of Defense for Program Analysis and 
Evaluation, ASD(PA&E) [Ref. 3]. Three major points are 
stressed by ASD(PA&E) : 
@ DOD managers should use COEAs to help understand the 
effects of different technological solutions in terms of 
military capabilities and payoffs. 


@ DOD managers should use realism in assessing program 
prospects, because future acquisition programs 











experiencing difficulties will face termination instead of 
DOD assistance. 


@® DOD managers should maintain the ability to adopt 
different courses of action while executing a program 
within budget totals. 

The COEA is a useful tool that is supposed to address all 
tixee points made by ASD(PA&E). It is prepared and considered 
at milestone decision reviews of all acquisition programs, 
beginning with Milestone I, Concept Demonstration Approval 
[Ref. 6). The COEA ideally aids decision making by 
illuminating each alternative’s relative cost and operational 
advantages and disadvantages, through comparisons of life 
cycle costs? and measures of operational capabilities. It 
also shows sensitivity to possible changes in key assumptions 
and variables. The COEA also attempts to facilitate 
communications among decision makers and staffs and document 
acquisition decisions by providing a historical record of 
alternatives considered at each milestone decision point. 

The COEA is comprised of several sub-analyses, as depicted 
in Table I [Ref. 6]. This thesis examines COEA cost, cost- 
effectiveness comparison, and sensitivity analysis areas for 
three P-3 aircraft alternatives tasked with conducting ASUW in 


a joint (multi-service), littoral (shallow water) scenario. 


lnife cycle costs are defined as the sum of the following 
costs: program flyaway costs (procurement cost of basic unit, 
recurring and non-recurring production costs, and recurring system 
project management); training and support equipment cost; cost of 
initial spares; research, development, test and evaluation costs; 
military construction costs; and operation and maintenance costs. 








The P-3 alternatives are a subset of the options that meet 
this particular DOD mission need. Other DON commands will 
examine other options, such as unmanned air vehicles or 
satellite based systems. 


TABLE I 
COEA SUB-ANALYSIS AREAS 


« 
1 
——— ee 
e 
. 
° . 
s 


Mission Need Analysis Identifies forces to meet specific 
operational need. 
| Threat Evaluation Describes projections of enemy threat 
over time. 
System Evaluates interoperability of system 
Interrelationships with current forces. 
Multi-Role Systems Evaluates ability of system to 
conduct different functions. 
Measures of A measure of operational capabilities 
Effectiveness (MOE) in terms of battle outcomes. 
Costs Measures resource inputs over system | 
life-cycle. | 
| 
| 


| 
| Cost -Effectiveness Examines marginal change in MOEs and 
| Comparison costs on an equal cost or equal 





effectiveness basis. 


Sensitivity Analysis Highlights effects of changes in 
threat, key performance criteria, or 
other baseline parameters. 





C. THE PROPOSED PROGRAM AND METHODOLOGY 

As a result of lessons learned in Operation Desert Storm 
and guidance promulgated in the Department of the Navy (DON) 
1992 White Paper "...From the Sea" [Ref. 7], a mission need 


has originated for additional ASUW platforms. The DON White 











Paper notes that our National Security Strategy has shifted 
from an emphasis on global threats and traditional independent 
blue water Navy roles to one focused on regional challenges in 
joint, littoral scenarios. It defines the heart of naval 
warfare as battlespace domination and states that ASUW, 
performed traditionally by battle groups, is an integral part 
of this mission. In light of declining defense budgets and 
reduced traditional assets, DON is exploring alternative ASUW 
platforms and mixes. By utilizing the COEA to examine mission 
costs and tactical effectiveness, DON can select the best 
alternatives to perform the ASUW mission. 

The COEA alternatives examined in this thesis are 
platforms currently being considered for the alternative ASUW 
role. The alternatives, depicted in Table II, all involve the 
Navy's Maritime Patrol Aviation (MPA) force composed of 
Lockheed P-3 Orion aircraft. 


TABLE II 
ASUW COEA ALTERNATIVES 





Alternative 1 Existing P-3s - the Status 
CURRENT P-3 FORCE Quo 
Alternative 2 Existing P-3s with Minor 
P-3 AIP PROGRAM Avionics Upgrade 
Alternative 3 New Production P-3 
ORION II PROGRAM | 









All alternatives are capable of conducting ASUW search, 


location, tracking, and attack operations in littoral 





scenarios and passing real-time information to a higher 
authority that is commanding joint forces. Each COEA 
alternative is evaluated using MOEs developed by NAIR-526, the 
Naval Air Systems Command (NAVAIRSYSCOM) Warfare Analysis 
Division [Ref. 8], and author-developed "analogous" life cycle 
cost estimates. 

The COEA’s sensitivity analysis concentrates on the 
effects of increased aircraft service life and cost variance. 
Service life effects are examined using two P-3 life extension 
programs, the Sustained Readiness Program (SRP) and the 
Service Life Extension Program (SLEP). Cost variance effects 
for the modification and new aircraft program are studied 


using data extracted from Selected Acquisition Reports (SAR)? 





by the author and RAND Corporation [Ref. 9]. 

Given the COEA requirement to evaluate the three P-3 
alternatives using the MOEs, cost estimates, and sensitivity 
analyses previously mentioned, an analytical tool is needed 
that compares each alternative while considering the overall 
effects of its choice on the entire P-3 fleet. The most 
effective, or optimum plan, would meet the COEA mission need 
and be readily supported by and integrated into the current P- 


3 force structure. 


2The SAR is used by Congress to monitor the cost of Major DOD 
weapons acquisition programs, and is required by Section 2432, 
Title 10, of the United States Code. SARs are governed by DOD 
Instructions (DODI) 5000.2 [Ref. 6] and 5000.2-M [Ref. 10]. 











MPAMOD1 (MPA MODernization model, version 1), a mixed- 


integer optimization model, is a tool that can accomplish this 
task. MPAMOD1 determines an effective multi-year plan for 
implementation of each P-3 alternative while establishing 
constraints to ensure, if possible, that: 

@ Required P-3 fleet inventory levels are maintained, 

@ Minimum ASUW MOEs are met, 

@® Maximum average P-3 fleet age is not exceeded, 

@® Program expenditures remain within budget limits, and 


® Minimum and maximum program production line limits are not 
violated. 


MPAMOD1 is a modified "production/inventory" model. A 
similar model, the "PHOENIX" model, was used to analyze the 
Army’s helicopter fleet modernization program [Ref. 11]. 
MPAMOD1 determines optimal schedules for aircraft modification 
and/or new production, and existing aircraft retirements. It 
also provides annual totals for budget expenditures, aircraft 
inventory levels, average P-3 fleet ASUW effectiveness, and 
average P-3 fleet age. MPAMOD1 provides the output necessary 
to evaluate each COEA alternative’s effect on the entire P-3 
fleet, providing the "best possible" implementation plan. 
Thus, a level "playing field" is established for all 
alternatives. 

After the best COEA alternative is determined, a second 
question is examined. The model is used to address the cost 


effectiveness of major aircraft modification programs, 








assuming that the modified aircraft service lives are not 
extended. A new production P-3 with a longer airframe service 
life is compared to modifying existing P-3s, which have 
shorter service lives. The modified P-3s have lower initial 
costs than the new aircraft, and both have the same mission 


systems installed. 


D. THESIS OUTLINE 

Chapter II reviews the background and history of the 
weapon system cost estimates, SAR-based sensitivity analysis, 
and optimization models similar to MPAMOD1. Chapter III 
describes the cost estimates for the three basic COEA 
alternatives and the two life extension programs that can 
modify the alternatives. The cost estimate for the P-3 
modification involved in the airframe service life analysis is 
also presented, and all estimates are examined for validity 
using historical program data. Chapter IV details the SAR- 
based sensitivity analysis and establishes a cost variance 
factor to improve cost estimate accuracy. Chapter V describes 
MPAMOD1 and its assumptions along with a description of input 
data required by the model. Chapter VI details the nine 
separate scenarios that are modeled and run to determine the 
best COEA alternative. The effects of the P-3 life extension 
programs and the alternatives’ sensitivity to changes in cost 
are also examined. In addition, the airframe service life 


analysis (major modification versus new aircraft) is 





presented. Chapter VII discusses conclusions, recommendations, 


and areas for further study. 














II. BACKGROUND AND HISTORY 


A. INTRODUCTION 

A comprehensive body of literature exists that addresses 
cost estimation techniques, cost estimate variance, and 
optimal resource allocation for weapon systems. This thesis 
builds on historical efforts dating back more than 30 years, 
predominately by government sponsored research organizations 
such as RAND and the Institute for Defense Analyses (IDA). 
Private contractors and academic institutions have also played 
their part. Their past efforts provide fertile ground to 
determine a method for estimating the minimum life cycle cost 
of the three P-3 ASUW COEA alternatives, and to improve 
subsequent estimate accuracy. 

The chapter opens by examining the literature associated 
with cost estimation techniques for aircraft modification and 
new aircraft procurement programs. Techniques for determining 
cost variance are described next, followed by a discussion of 


the history of the optimization model. 


B. WEAPON SYSTEM COST ESTIMATION TECHNIQUES 

The literature was examined to determine which of the 
three common cost estimation techniques, "parametric", 
"analogous", or "engineering" (all defined below), would be 


best for costing the P-3 avionics modifications and new 


11 














aircraft alternatives for the COEA. Ideally, COEA cost 
estimates should be accurate enough to determine relative cost 
differences between numerous alternatives despite the lack of 
detailed information available in a program's early stages. 
Parametric cost estimation techniques meet the criteria stated 
above, and are defined by Michael G. Sovereign as follows: 
The top down [parametric] approach has the 
advantage of being available early when 
decisions on configuration of the product are 
still being made, i.e., in the design stage. 
It uses statistically estimated, logical 
relationships between the cost per unit of 
product and the physical and performance 
characteristics (or parameters) of the 
product, i.e., weight, speed, etc. [Ref. 12] 
Parametric approaches seem ideally suited for the COEA cost 
estimation process, as opposed to the analogous or engineering 
techniques, and the vast quantity of literature available 
primarily addresses this technique. 

Sovereign defines the analogous method as an early, crude 
estimate that is made by picking the closest existing analog 
to the proposed system [Ref. 12]. Analogous cost estimates can 
be generated from historical cost data for entire systems or 
from task elements of previous systems. They require keen 
judgment by the cost analyst, because correction factors must 
be developed and applied to successfully compare the historic 
costs of older systems to new systems. 


Finally, the engineering method is defined as: 


[An] approach...only available from the detailed input 
calculations of the industrial engineers and _ cost 


12 











accountants many months or years after the initial cost 
estimates are made. [Ref. 12] 






Because engineering cost estimates require large historical 






data bases and long preparation times, they are unsuitable for 







early COEA cost estimates. The literature search, therefore, 


focuses on parametric and analogous models. 






1. Aircraft Modification Cost Estimation Techniques 





There have been numerous studies, undertaken primarily 






by the RAND Corporation, addressing the problems of aircraft 






modification cost estimation. In 1978, RAND evaluated nine 







existing life-cycle cost preliminary planning models for the 


United States Air Force to determine their effectiveness in 






evaluating aircraft modification costs. The RAND study 





concluded that the models had many shortcomings and were of 







limited use (Ref. 13]. 


In two separate 1981 studies, RAND attempted to 







develop parametric Cost Estimating Relationships (CERs) for 


both avionics systems and aircraft structural modifications. 






The pursuit of avionics CERs proved largely unsuccessful [Ref. 






14], as did the search for aircraft structural modifications 






CERs [Ref. 15], which proved unreliable when compared to the 






actual costs of known modification programs. In each case, 






however, the RAND models yielded improvements over then- 





current estimation techniques. Despite lack of major success, 






the avionics CERs were more accurate than an earlier "cost per 






pound method" and the aircraft modification CERs proved useful 











when applied with discretion and an understanding of original 
production history. 

In 1987, the General Accounting Office (GAO) [{Ref. 16] 
issued a report pertaining to the United States Air Force’s 
B-52, A-10, and F-111 modification programs. The report noted 
that the programs were over funded due to high original 
modification cost estimates. The B-52 program, for example, 
accumulated $323 million in excess funds alone by obligating 
only 86% of total funds appropriated for modifications. DOD 
and Air Force officials justified the anomaly as follows: 

® Original estimates are based on contractors’ "rough order 
of magnitude" estimates, with more precise estimates 
occurring after the program’s first-year budget requests 
are approved. 

® The procurement process results in cost reductions. 

@® Modification scope may be restricted. 

@® Modification costs are padded up to 30 percent to account 
for risk associated with concurrent production and 


research and development. 


@® Air Force estimates tend to be high to preclude the 
necessity and delay of requesting additional funds. 


The GAO study is evidence that accurate modification cost 
models did not exist at the end of 1987. 

In 1990, the Boger and Liao Aircraft Modifications 
Cost Analysis, funded by the NAVAIRSYSCOM Cost Analysis 
Division, NAIR-524 [Ref. 17] attempted to develop CERs for 
aircraft modification programs. The comprehensive, multi-year 
study was started to assist NAVAIRSYSCOM in accurately 


estimating aircraft modification program costs, which were 


14 














approaching $2 billion annually in DON. The study discussed 


the current DON policy of pursuing aircraft modification 
efforts instead of procurement. The study also noted the need 
for modification cost models due to the increased use of this 
method to upgrade DON aircraft in the austere funding 
environment of DOD. 

Boger and Liao noted that NAVAIRSYSCOM did not possess 
an aircraft modification cost estimation model, and this 
impeded execution of planning studies and COEAs. Both audits 
of NAVAIRSYSCOM by the DOD Inspector General and existing DON 
Instructions document the need for this type of model to aid 
aircraft modification planning and management. 

Boger and Liao discussed four factors that affect 
aircraft modification cost model development: 

@® Modifications vary in complexity. 
@® Modifications vary by individual aircraft. 


@® Various methods and organizations are used to accomplish 
modifications. 


@® Organizations that accomplish modifications have different 
levels of prior experience and facilities. 


The Boger and Liao CER effort proved completely unsuccessful, 
primarily due to the unique character and tailored nature of 
each modification, and the four factors above [Ref. 17]. 

The preceding literature review leads to the 
conclusion that aircraft modification program CERs did not 
exist as of March 1993, ruling out use of the parametric cost 


estimation technique. Since engineering cost estimates are 


15 








unsuitable for early COEA analyses, this thesis uses the 
analogous cost estimation method as the principal technique 
for estimating the P-3 avionics modification costs required 
for the COEA. 
2. New Aircraft Cost Estimation Techniques 

RAND Corporation has also been the leader in 
developing CERs for new aircraft airframes. In 1972, RAND 
generated a long-range cost estimation planning model that 
predicted the cost of military airframes [Ref. 18]. The model, 
using multiple regression analysis, related cost or man-hours 
to aircraft physical and performance’ characteristics, 
essentially aircraft unit weight and speed. 

In 1976, RAND produced a further study of airframe 
CERs for ASD(PA&E) [Ref. 19]. The new model attempted to 
address existing user concerns with the 1971 model, including 
lack of additional explanatory variables, individual aircraft 
classes, and changes in airframe structural materials. The 
model proved only partially successful. Aircraft unit weight 
and speed were still the most statistically significant 
explanatory variables. Of all structural materials examined, 
only construction involving aluminum provided sufficient 
information to draw conclusions about the impact of structural 
changes on cost. 

The last study, conducted by RAND in 1987, updated and 


extended the CERs developed in 1976 [Ref. 20]. Additionally, 
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the study divided the full estimating sample into subsets 
representing major aircraft types, and examined the 
explanatory power oof program structure and airframe 
construction techniques. Unfortunately, the subset most 
applicable to this thesis, the bomber and transport subset, 
did not yield a single acceptable CER for any individual or 
total program cost element. Attempts to incorporate subsets, 
program structure and airframe construction techniques were 
considered unsuccessful. The RAND study recommended that 
estimates be developed by analogy or by using the equation set 
developed by aggregating data from all aircraft mission types. 
The study concluded that the most representative equation set 
still used aircraft empty weight and speed as the designated 
explanatory variables, repeating the conclusions of the 1972 
RAND report [Ref. 18]. The RAND CERs produced by Hess and 
Romanoff are used in Appendix A to validate the new P-3 cost 


estimate developed using the analogous method. 


C. COST VARIANCE ESTIMATION TECHNIQUES FOR COST ESTIMATES 
Cost estimates are future predictions subject to 
uncertainty and inaccuracies. Once a program’s original cost 
estimate is developed, it changes over time as new information 
becomes available. A method is needed to quantify these 
program cost estimate changes, or variances, which are defined 
as the difference between a weapon system’s original and 


current cost estimates. The original cost estimate can be 
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adjusted by identifying cost variances which are categorized 
as cost gruwth or cost reductions. 

For example, assume that costs for a particular program 
have grown 15% over the original program cost estimate. If the 
.cost variance could have been estimated by some method, the 
original estimate could have been adjusted as follows: 
((Original Cost Estimate) x (Cost Variance/100)] + [Original 
Cost Estimate] = [Adjusted Cost Estimate]. 

This thesis uses analyses of DOD SAR data to determine 
cost variance. This is one approach among many that can be 
used to determine cost variance. 

The DOD SAR measures cost variance using the concept of 
the baseline. According to Tyson of IDA, one type of baseline, 
the development baseline, can be defined as: 

...-the estimates of technical, schedule, and 
cost goals at the time that the program 
entered full-scale development (FSD). 
Technical goals...cover the weapon system’s 
performance and technical characteristics... 
schedule goals...include dates for FSD 
contract award, initial operating capability 
(IOC), and the various acquisition milestones. 
The cost goals include costs for development, 
production, military construction, and other 
program costs in both current and constant 
dollars. Each program has an establi.shed base 
year, typically the FSD year. [Ref. 21) 
This thesis, therefore, is primarily interested in SAR cost 
variance, and how it changes over time with respect to a 
selected baseline. By quantifying those changes, the data can 


be used to attempt to improve cost estimate accuracy. 
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1. Single Program Measures 

Management Consulting and Research, Inc. (MCR), a 
private contractor performing an analysis for ASD(PA&E), 
formulated a simple cost variance measure in 1981 to quantify 
DOD acquisition program cost growth [Ref. 22]. The measure, a 
ratio of the difference between the current estimate (CE) and 
the development estimate (DE), divided by the DE, was measured 
in current year dollars. Disadvantages included a lack of 
adjustment for quantity or inflation. 

In 1979, RAND released a study that stressed a 
different method of quantity adjustment [Ref. 23]. Cost 
variance, calculated in base year dollars, was measured as a 
ratio of CE to DE. RAND adjusted quantity back to DE quantity 
(avoiding the "floating baseline" problem by essentially 
holding quantity constant) by subtracting the SAR quantity 
variance from the CE. RAND then normalized the variance along 
the program’s cost-quantity curve, which is an improvement 
curve that implies a non-linear reduction in unit costs as 
quantities increase.? 

In 1989, an IDA report evaluated major system cost and 
schedule trends and acquisition initiative effectiveness [Ref. 


24]. Approach advantages included separation of development 


3The cost-quantity curve is synonymous with the "learning" 
curve, a function defined as follows: Variable cost of the Qth 
unit = [Variable cost of the first unit] x [(Cumulative Quantity) * 
(Learning Index)], where the learning index is negative. Typical 
learning index numbers range from -0.70 to -0.95. 
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and production cost growth and a new quantity adjustment 
method. IDA, believing that SAR variance category data was 
unreliable, disregarded it entirely. Instead, price- 
improvement curves were developed independently from SAR 
annual data for completed program production years. The cost 
of the originally planned development estimate quantities was 
calculated using these curves. 

In 1991, Bliss of OSD(PA&E/EA&RPD) reported on the 
analysis of data assembled by RAND in an ongoing study of cost 
growth [Ref. 25]. The report discussed characteristics and 
causes of SAR cost growth, validity of SAR cost data, and data 
normalization techniques for 27 of 107 potential systems that 
submitted SARs. To reduce the scope of the study, the systems 
evaluated were the best and the worst performing systems as 
measured by the RAND-derived aggregate cost growth factor. 
They were spread over six commodity classes: missiles, combat 
aircraft, combat vehicles, ships, helicopters, and 
electronics. 

Bliss normalized the data for inflation and quantity, 
using a normalization procedure developed at RAND. The 
following results were reported: 

@® Defense cost growth is more modest than commonly assumed 
and generally more modest than cost growth experienced in 


comparable economic sectors, such as that experienced when 
building large industrial power plants. 


® Cost growth from discretionary sources, such as decisions 
external to program’s defined Milestone II baseline, is 
twice that of growth associated with "estimation error." 











These decisions include changes in system capability and 
acquisition strategy changes. 


@ Many factors previously cited as sources of cost growth 
(concurrence, failure to achieve program planned 
procurement rates, "turbulence", etc.) are not powerful 
explanatory variables in themselves. 

@® The most important explanatory variables are program size 
and commodity class, with size inversely related to cost 
growth (smaller programs incur greater proportional cost 
growth than larger programs). 

In 1992, Hough of RAND documented pitfalls associated 
with calculating cost growth using SAR data [Ref. 26]. The 
RAND note observed that many changes had been made over time 
to improve the quality of SAR data. Even so, the following 
notable problems remain: 

@ Failure of programs to use consistent baselines, 

@ Exclusion of some significant cost elements, 

@® Exclusion of special access programs, 

® Constantly changing preparation guidelines, 


@ Inconsistent interpretation of preparation guidelines 
across programs, 


@ Unknown and variable funding levels for the program 
manager’s program risk fund, 


@® Cost sharing in joint programs, and 


@® Reporting of cost change effects instead of their root 
causes. 


In order to cope with these problems, the cost analyst using 
SAR data must apply adjustments, realizing that not all 
effects are correctable. 

Two problems that are correctable using accepted 


analytical approaches are inflation and changes in quantity. 
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According to Hough, the quantity changes are adjusted to the 
originally estimated quantity using two methods: 


The simplest method extracts the amount the 
SAR reports for quantity and adjusts’ the 
current estimate accordingly. More 
sophisticated methods involve an adjustment 
based on the program’s total cost-quantity 
curve. When quantity has changed frequently 
and by a large margin, the method used and the 
care taken to fully capture all costs related 
to the change can result in strikingly 
different measures of cost growth for the same 
program. [Ref. 26] 


Hough’s note concludes by stating that SAR data is suitable 
for identifying broad-based trends and temporal patterns 
across a wide range of programs. 
2. Multiple Program Measures 
In 1980, IDA examined cost growth for multiple 


programs over two time intervals, DE to Initial Operational 


Capability (IOC), and IOC to latest available estimate [Ref. 


27). The approach used by IDA has the following advantage: 
@ Separation of past and future program growth, 
@ Adjustment made for inflation, and 


@ Development and production cost growth was measured 
separately and in total. 


The principle disadvantage of the IDA approach is the lack of 
a quantity adjustment. 

In the IDA report issued in 1989, program aggregation 
was done differently [Ref. 24]. Program inclusion was 
dependent upon maturity. Programs having fewer than three 


years of experience past a particular baseline were not 











included. Also, total program cost growth was calculated using 
a weighted average method based on program size in base year 
dollars. 

In 1992, Drezner of RAND, in work ongoing for the 
United States Air Force, analyzed weapon system cost growth 
for 197 major weapon systems reporting in the SAR system as of 
December 1990 [Ref. 28]. Drezner normalized the data for 
inflation and quantity effects, and used a weighted average 
cost growth method to account for program size when 
aggregating the data. Program maturity effects were accounted 
for by only including programs three or more years past full 
scale development start. All cost growth was referenced to and 
measured from a specific baseline, i.e., cost growth was 
measured from development and production baselines separately. 

Drezner’s objectives were to quantify the magnitude of 
the cost growth problem and identify factors affecting cost 
growth. Drezner discovered that cost estimates were 
systematically biased towards underestimation, with 
approximately 20% cost growth experienced at both planning and 
development baselines, decreasing to 2% at the production 
baseline. There was no apparent improvement over time; cost 
growth has fluctuated around 20% since the mid 1960s. 

In addition, the following was discovered with respect 
to factors involving cost growth: 

.we examined many possible explanatory 


variables, including macro level development 
strategies, schedule related factors, and 
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management and budget considerations. ..We 

found few strong relationships. ..While program 

length, program size, Maturity, and 

modification versus new developments are 
significant correlations, no single factor 
explains a large portion of the observed 
variance in cost growth outcomes...there is no 

"silver bullet" policy response to...cost 

growth. [Ref. 28] 

Drezner concludes by suggesting that any policy solution will 
be complex, incorporating all aspects of the acquisition 
process and requiring changes in behavior of all responsible 
parties. He states that the sum of current DE program baseline 
cost estimates is $450 billion, and if 20% ($90 billion) could 
be considered significant, then correcting the problem 
warrants the effort involved. 

In 1993, Drezner of RAND, in work forthcoming for 
ASD(PA&E), describes the Defense System Cost Performance 
Database (DSCPD) [Ref. 9]. The report’s goal is to provide a 
"living" data base for use by analysts both in and out of 
government to improve understanding of the weapon system cost 
growth problem. Limitations and caveats used while creating 


DSCPD are documented, and the data base is internally 


consistent. The DSCPD is projected to be updated annually. 


D. OPTIMIZATION MODEL 


MPAMOD1 is a mixed-integer optimization model [Ref. 29] 


that is a variant of common production/inventory planning 


models [Ref. 30]; the major difference is that MPAMOD1 does 


not incorporate external demand. Instead of external demand, 








aircraft procurement and/or modifications are driven in 
MPAMOD1 by inventory and fleet effectiveness requirements. 

MPAMOD1 stems from the "PHOENIX" optimization model used 
to modernize the U. S. Army’s helicopter fleet in the late 
1980s (Ref. 11]. The PHOENIX model 

...-Captured complex procurement and modernization 

tasks in an optimization-based decision support 

system... [that] recognizes yearly operating, 
maintenance, service-life extension, and new 
procurement costs while enforcing constraints on fleet 
age, technology mix, composition, and budgets overa 
multi-year planning horizon. ..PHOENIX has been adapted 

to tactical wheeled vehicles and is under 

consideration for further applications. [Ref. 11] 

The PHOENIX model was tailored for long-range planning, ata 
high level of detail, for capital equipment procurement, use, 
repair, and retirement. Individual helicopter programs were 
optimized in the larger context of their impact on overall 
Army helicopter fleet effectiveness. 

Two Naval Postgraduate School Masters theses, written in 
1990 and 1993, applied the methodology of the PHOENIX model to 
MPA fleet modernization. The first thesis, by Drash, developed 
a model (referred to as the "Drash model") that is outdated 
because it does not include the proper mix of current MPA 
modernization and life extension programs [Ref. 31]. The 
second thesis, by Osborn, developed a model (referred to as 
MPAMOD) that implemented all current MPA programs [Ref. 32]. 
MPAMOD1 incorporates minor modifications to Osborn’s MPAMOD. 


MPAMOD1 produces output that delineates annual program 


life cycle costs, program executability within budget 
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constraints, and preliminary aircraft modification scheduling 
over a long term program planning horizon. It provides 
necessary information to optimize P-3 COEA alternatives in the 
larger context of their impa..: on overall P-3 fleet 
effectiveness. In short, MP‘MOD1 rrovides planning information 
required to select the best COEA alternative, and gives a 


"first cut” answer to solution implementation. 


E. CONCLUSIONS 
After examining the literature, the following conclusions 
can be drawn: 


CERs do not exist for aircraft modification programs. 


CERs do not exist for the subset of bomber and transport 
new production aircraft. The CERS recommended for use 
involve aggregation of all aircraft mission type data. 


A methodology exists to determine program cost variance 
using SAR data. Applying the calculated cost variance is 
one method of improving cost estimate accuracy. 


The COEA’s use of a mixed integer optimization model to 
optimize resource allocation provides information needed 
to select and implement the best COEA alternative. 


These conclusions result in the following effects for this 
thesis: 

@ Cost estimates are calculated using the analogous method. 
The RAND CER equations produced by Hess and Romanoff are 
used only to validate the cost estimate for the new 
production P-3. 


® The cost variance analysis uses data generated by the 
author and by the RAND DSCPD. 





III. COST ESTIMATES FOR COEA ALTERNATIVES 


A. INTRODUCTION 

The first COEA alternative’s cost estimate, the status 
quo, is not required because all prior production costs for 
current fleet P-3s are sunk costs. Operations and Maintenance 
(O&M) costs for all alternatives are addressed in Chapter V. 

All other cost estimates are developed using the analogous 
method. The analogous cost estimates are less accurate than 
desired because of limited access to original source cost 
data. Detailed cost estimate calculations are provided in 
Appendix A. The estimates are then compared to historical cost 
data contained in Appendix B to judge estimate validity. 

Cost estimates for two P-3 life extension programs are 
also included. The costs are required for certain COEA 


scenarios that evaluate P-3 service life extension effects. 


B. COST ESTIMATE FOR P-3 ASUW IMPROVEMENT PROGRAM 
The P-3 AIP is being rapidly implemented through use of 


the acquisition concept of "streamlining", defined as follows: 


...-it is the policy of DOD to use commercial and other 
nondevelopmental items to the maximum extent 
practicable in procurement of supplies...the policy is 
designed to promote efficiency in the use of taxpayer 
resources to procure supplies and provide timely and 
effective support for the armed forces. [Ref 33] 








This concept requires the following key assumptions to be made 
concerning the modification cost estimate: 


@® The design uses current off-the-shelf (COTS) non- 
developmental item (NDI) systems. 


® The COTS/NDI systems are all flying or have previously 
flown. 


@ No initial RDT&E is required, although follow-on test and 
evaluation (FOT&E) is needed. 


@ All hardware initially operates as stand alone systems, 
with full P-3 software integration deferred for later 
preplanned product improvement (P3I) efforts. 

® The AIP modification is installed in a baseline P-3C 
Update III aircraft that has a ASQ-212 (CP-2044) Data 
Processing System (DPS), Global Positioning System (GPS), 
and ALR-66(V)3 Electronic Support Measures (ESM) System. 

These are valid assumptions because the systems that comprise 
the P-3 AIP are currently flying in specially modified P-3 
"Outlaw Hunter" aircraft or on other DOD aircraft. Only 
minimal software integration is contemplated for the AIP, and 
no RDT&E is presently funded. 

In general, avionics modifications are procured as "kits" 
and installed in applicable aircraft by commercial contractors 
or government repair facilities, such as Naval Aviation Depots 
(NADEPs). Modification "A" kits consist of airframe and 
installation components. Modification "B" kits include all 
avionics equipment. For purposes of standardizing the cost 
estimate summary table, "A" kit costs are placed in the 
"Airframes/Changes" category and "B" kit costs in the 


"Electronics/Comm" category. These are arbitrary groupings 


used for comparison and evaluation purposes only. 


28 

















1. System Description 
The P-3 AIP’s new sensor subsystems ‘nclude an Inverse 
Synthetic Aperture Radar (ISAR), improved Electronic 
Surveillance Measures (ESM) with spinning direction finding 
(DF) antennae, Infrared Detection System (IRDS) improvements, 
and an electro-optical surveillance system. 


TABLE III 
P-3 ASUW IMPROVEMENT PROGRAM SUBSYSTEMS 


—— - — 
| CATEGORY SUBSYSTEM 


- APS-137(V)5 ISAR Radar 
- ULQ-16(8.0) ESM Mod 

- ESM Spinning DF Antenna 
AAS-36 IRDS Improvement 
Electro-Optical System 


OTCIXS 
- TRE 

- DAMA SATCOM Secure Voice 
ICS Modifications 


(3) CHRDs, (1) PCHRD 
- (3) PEPs, (2) DEP 

Keyboards and Trackballs 
Hard Copy Recorder 


























COMMUNICATIONS 






















DISPLAYS AND CONTROLS 









Fuel Tank Foam 
- Missile Warning and 
Countermeasures Provisions 


1 CENTRAL PROCESSING SYSTEM - Minimum Hardware Mods 
| Se - Minimum Software Mods 


New communication subsystems include Officer in Tactical 





SURVIVABILITY 
i 
| 


Command Information Exchange System (OTCIXS) , Tactical Receive 
Equipment (TRE), Demand Access Multiple Address (DAMA) capable 
Satellite Communications (SATCOM) secure voice, and Intercom 


Communications System (ICS) improvements. Three sets of Color 
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High Resolution Displays (CHRD) with associated Programmable 
Entry Panels (PEP) are installed at the Tactical Coordinator 
(TACCO) , Navigation/Communicator (NAVCOMM), and Sensor Station 
3 (SS3) stations. A Pilot Color High Resolution Display 
(PCHRD) and associated Data Entry Panel (DEP) is installed in 
the cockpit. The P-3 AIP systems are summarized in Table III. 
2. Cost Estimate 

The P-3 AIP costs are expressed in constant FY 1993 
dollars using March 1993 deflators obtained from the Navy 
Center for Cost Analysis (NCA). The detailed cost estimate 
methodology (Appendix A) uses an analogous approach to 
generate modification kit costs. The kit costs are then 
multiplied by the yearly procured quantity and added to non- 
recurring engineering (NRE) costs to calculate total program 
flyaway cost. Support equipment, training equipment, and 
spares are added to determine program acquisition cost. 

Modification kit unit costs remain constant over 
program life. Omission of the role of learning in this 
modification program is justified as follows: 

@® The program’s acquisition and contracting strategy 
requires COTS/NDI subsystems purchased by "piggy-backing" 
onto existing government contracts. Other government 
agencies/programs have already reaped the benefits of any 


subsystem learning. 


@ Relatively minor airframe and internal cabin modifications 
are required. 


@® Appropriate inflation factors have been applied to 
previously purchased kit items. 
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The P-3 AIP cost estimate is summarized in Table IV. 


TABLE IV 
P-3 ASUW IMPROVEMENT PROGRAM COST ESTIMATE 
(THOUSANDS OF FY93 DOLLARS) 


CATEGORY TOTAL COST 










UNIT COST 























Airframe/Changes 146,540 

NRE 35,000 
Engines/Access 0 
Electronics/Comm 327,624 
Armament /Other Q 














Total Flyaway 509,164 509,164 7,488 
Ground Support Eq 43,188 
Training Eq/Other | 120,306 
Weapon Sys Cost 163,494 9,892 






Initial Spares 
Procurement Cost 


RDT&E 
Military Const 






76,923 
76,923 749,580 11,023 

0 

—— = 9 
) 749,580 11,023 


C. NEW PRODUCTION P-3 COST ESTIMATE 

The COEA new production aircraft alternative, the P-3 
ORION II, is a new airframe based on the existing P-3 design. 
The program’s acquisition strategy is to procure the new 
aircraft as a reprocurement of an existing design, requiring 
the following cost estimate assumptions to be made: 


@® The ORION II is an Acquisition Category 1D program that 
enters the procurement process at Milestone IV. 


@® The contract is let sole source to Lockheed to take 
advantage of existing P-3 tooling and expertise. 


® Risk is considered low because al! systems are NDI. 
@ The buy will take advantage of the existing production 


line from a foreign procurement, but a gap will exist 
before Navy production begins. 


31 








@ Because the contract is sole source, competition is 
maximized at the subvendor level. 


1. System Description 
The Orion II program meets operational requirements 
drafted January 1991. Those operational requirements include: 
@® Range/endurance of 1600 navtical miles/4 hours on station, 
@® A 5000 pound payload with provisions for future growth, 
® Capacity for 120 sonobuoys, 
® Weapons upgrades, and 
® Inflight refueling. 
The ORION II systems are summarized by category in Table Vv. 


TABLE V 
ORION II SYSTEMS 


CATEGORY 


AIRFRAME 
and Takeoff Gross Weight 
New Engines 
Improved Aux Power Unit 
Pressurization Improvements 
Fuel Tank Foam 
Survivability Provisions 
Inflight Refueling 
Anti-skid and Carbon Brakes 


AVIONICS Update III Plus Systems 
AIP Systems 
Digital Magnetic Anomaly 
Detection (MAD) Sensor 
Color Weather Radar 
Ring Laser Gyro Inertial 
Digital Fuel Quantity 


MK-50 Torpedo 
Provisions for 120 
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2. Cost Estimate 

The P-3 ORION II new production aircraft program 
procures 68 aircraft over a seven year period. All costs are 
expressed in constant FY93 dollars using the 1993 NCA 
deflators, but aircraft unit costs do not remain constant over 
program life. A learning curve of 90% is assumed for this 
aircraft procurement program based on historical and projected 
contractor performance*. The P-3 ORION II detailed analogous 
cost estimate is given in Appendix A. The cost estimate is 
summarized in Table VI. 

TABLE VI 


P-3 ORION II COST ESTIMATE 
(THOUSANDS OF FY93 DOLLARS) 









i 








: 
lEngines/Access 
j#Electronics/Comm 
yArmament /Other 
/ Total Flyaway 









5,326,300 78,328 






463,100 










iTraini 695,200 | 
1,158,300 6,484,600 95,361 

Initial Spares ___375,500 | 

Procurement Cost 375,500 6,860,100 100,884 | 

| 195,600 | 

i 


103,760 


’ 
U 








Se ES BSL ae 


195,600 7,055,700 


*variable cost of the Qth unit = [Variable cost of the first 
unit] x [(Cumulative Quantity Q)* (-0.90)] 
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D. COST ESTIMATE FOR P-3 UPDATE IV 

The P-3 Update IV program was conceived and executed in 
the mid-1980s to aid P-3 efforts against enemy fifth 
generation submarines. Primarily an Antisubmarine Warfare 
(ASW) improvement program, it was cancelled in 1992 as the 
submarine threat abated. It is considered in this thesis 
because it is more than three times the size of the P-3 AIP 
modification program and therefore provides a more 
representative program than that program for use when 
comparing the cost effectiveness of aircraft modification 
programs to new aircraft procurement programs. The procurement 
and modification programs are assumed to yield different 
service lives, with the aircraft modification program yielding 
the shorter service life. The system description is provided 
in Appendix A. 

The P-3 Update IV program modifies 68 aircraft over a 
seven year period by incorporating a modification kit. All 
costs are expressed in constant FY93 dollars using 1993 NCA 
deflators. Modification kit unit costs do not remain constant 
over program life. A 90% learning curve is assumed based on 
the following: 

@® program size, 


@® extensive airframe and internal cabin modifications, and 


@® new acoustic equipment unique to Update IV. 











The P-3 Update IV detailed analogous cost estimate is 
available in Appendix A. The cost estimate is summarized in 
Table VII. 

TABLE VII 


P-3 UPDATE IV COST (P-3 UIV) ESTIMATE 
(THOUSANDS OF FY93 DOLLARS) 


ee 
CATEGORY TOTAL COST UNIT COST 

477,156 

147,000 

0 

858,466 
Armament /Other ee * | 
Total Flyaway 1,482,622 1,482,622 21,803 
GSE/Trng Eq/Other 585,630 
Weapon Sys Cost 585,630 2,068,252 30,415 
!Initial Spares _223,000 
; Procurement Cost 223,000 2,291,252 33,694 


| RDT&E 319,000 
}Military Const Q 
j_Program Acq Cost 319,000 2,610,252] 38 386 Ed 

















Airframe/Changes 
NRE 

1 Engines/Access 

| Electronics/Comm 





























E. COST ESTIMATES FOR P-3 LIFE EXTENSION PROGRAMS 
Two P-3 life extension programs are examined for the P-3 
COEA, the Sustained Readiness Program (SRP) and the Service 
Life Extension Program (SLEP). The SRP’s purpose is: 
...to preemptively replace airframe components 
and systems identified as having potential for 
significant impact on future aircraft 
availability due to excessive time to repair, 
obsolescence, component manufacturing lead 
time or cost impact. [Ref. 34] 
The SLEP is designed to extend P-3 fatigue life by replacing 
fatigue critical components. Whereas the goal of the SRP is to 


capture 100% of aircraft fatigue life, the goal of the SLEP is 
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to extend an aircraft’s fatigue life beyond 100%. The model 
currently retires P-3s without life extensions at 30 years. 
SRP and SLEP modified P-3s are retired 8 and 10 years after 
the modifications are installed, respectively. 

The estimates were developed using information supplied by 
PMA-290. All costs are in constant FY93 dollars calculated by 
using NCA deflators. 

1. THE P-3 SUSTAINED READINESS PROGRAM 

The P-3 SRP is an airframe and selective equipment 
replacement program to renovate 193 of the 247 existing P-3C 
aircraft, or 78% of the fleet. On the basis of historical 
data, the average life extension realized from capturing 100% 
of aircraft fatigue life is eight years. The P-3 SRP cost 
estimate is summarized in Table VIII. 

TABLE VIII 


P-3 SUSTAINED READINESS PROGRAM COST ESTIMATE 
(THOUSANDS OF FY93 DOLLARS) 


1,367,136 
NRE 6,812 
Engines/Access 0 
Electronics/Comm eG 
Armament /Other 
Total Flyawa 1,373, OECERTT 1,373,948 


Support/Trng Eq ___42,989 
Weapon Sys Cost 42,989 1,416,937 7,342 
J—2e398 a ase 


Initial Spares 
Procurement Cost 1, J—2e398 a ase 1,418,136 7,348 


RDT&E 
Military Const 
_ Program Acq Cost = _1,418,136 | 
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2. THE P-3 SERVICE LIFE EXTENSION PROGRAM 
The P-3 SLEP is still in the preliminary stages of 
program formulation. Preliminary discussions with program 
officials [Ref. 35] have indicated the SLEP target unit cost 
goal to be $3.5 million (FY93). The SLEP modification extends 


aircraft service life ten years. 


F. COST ESTIMATE VALIDATION 

The cost estimates for the three COEA alternatives are now 
compared, for validation purposes, to historical cost data 
taken from Appendix B. The method used involves comparing unit 
flyaway costs and using ratio analysis of various cost 
categories (ratios of total support, spares, and RDT&E costs 
to total flyaway costs). A summary of the comparison data, 
obtained from Appendix B and Chapter III, is presented in 
Table IX. 

TABLE IX 


P-3 HISTORICAL AND CURRENT COST ESTIMATE COMPARISON 
(MILLIONS OF FY93 DOLLARS) 


cost “srrmare/| P-3 UIII|P-3 UIII|/P-7A P-3 UIV 
oe oe OR RATIO }1984 SAR/1989 SAR 


TOTAL FLYAWAY 1236.6 |5207.8 1482.6 
7 


| 
louarrrry | a2 | zs | 6s | ce | co | [es | 
} 


{UNIT FLYAWAY 41.7 85.7 7.5 21.8 78.3 
COST 


Fo ow ees eo eo eo 
a ee ee es ee 
pore ravzo | 228] ce | rox | ase | - faze | ae | 



















First, unit flyaway costs for the ORION II COEA 
alternative and the four historical aircraft are compared. The 
ORION II’s unit flyaway cost is much greater than the two 
early P-3C UIIIs and the later P-7 aircraft, and more closely 
resembles the P-11 estimate generated from Hess and Romanoff’s 
1987 CER data, discussed in detail in Appendix B. This can be 
attributed to the numerous structural changes envisioned for 
the ORION II airframe, which is different from previous patrol 
aircraft designs. It is more similar to a new aircraft than to 
a previous or existing weapon system, and the data reinforces 
this fact. The ORION II unit flyaway cost is reasonable in 
this light. Second, ratios for total support equipment, 
spares, and RDT&E costs as a percentage of total program 
flyaway cost are compared. The P-3 AIP is examined initially, 
followed by the ORION II and Update IV programs. 

The P-3 AIP support ratio is reasonable, but the spares 
ratio of 15% seems high. The preponderance of new systems in 
the AIP design and their unique sparing needs explains this 
anomaly. The historical data is weighted towards lower spare 
ratio percentages due to system commonality with previous 
designs, requiring fewer unique sparing requirements. The 
simple answer is that if the spare parts do not already exist 
in the supply system in quantity, as was the case in earlier 
P-3 modification programs, the program must buy them. 

The Update IV support ratio (39%) and spares ratio (15%) 


also seem higher than normal. This is explained by the 
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preponderance of new advanced ASW systems, and the need to 
support and spare them accordingly. 

When compared to the historical data in Table IX for 
similar systems, the ORION II’s support (22%), spares (7%) and 
RDT&E (4%) ratios are very similar, confirming the validity of 
these costs for the cost estimate. Initial spares procurement 
is greater than that of previous Update III programs, but less 
than the P-7A program. This is explained by the greater 
commonality of ORION II systems with the current P-3 fleet, 
comprised mainly of P-3C UIIIs. This also explains ORION II’s 
low RDT&E ratio. 

In conclusion, the ORION II flyaway cost, and the COEA 
alternatives and the P-3C Update IV support, spares, and RDT&E 
cost estimate ratios seem reasonable in light of program 


characteristics. The cost estimates pass validity checks. 


G. CONCLUSIONS 

Cost estimates for the COEA alternatives and the P-3C 
Update IV have been generated from detailed analogous cost 
estimates given in Appendix A. They have been compared to 
costs for similar historical systems and are reasonable. 

Recognizing that cost estimates are subject to 
uncertainties, a cost variance analysis using data obtained 
from SAR documents is conducted next. The analysis results are 
applied to the current cost estimates to improve their 


accuracy. 
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Iv. COST VARIANCE ANALYSIS 


A. INTRODUCTION 

After initial cost estimates are generated and their 
validity checked by comparison to previous historical cost 
estimates, some effort should be made to improve their 
accuracy. This is known as cost variance analysis. The SAR- 
based cost variance analysis is one of many such methods that 
can be used. 

A cost variance that is either cost growth or cost 
reduction is estimated from actual historical performance for 
similar acquisition programs. The data to estimate this 
variance is found in DOD SAR documents. It is applied to the 
original cost estimate as follows: 

[Adjusted Cost Estimate] = [(Original Estimate) x (1 + 
Variance) ]. 

This chapter first defines cost variance. It then outlines 
an author-developed five step method similar to the RAND 
method [Ref. 28] for calculating cost variance. These 
calculations are then made, followed by a validation of the 


results, and chapter summary. 


B. THE DEFINITION OF COST VARIANCE 
The cost estimate sensitivity analysis is based on the 


concept of "cost variance". Otegui defines cost variance as: 
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...the difference in cost between the original and 
current estimate of a given weapon system...it carries 
no a priori implication of inefficiency even when the 
variance is an increase: the variance may well be--and 
in fact, often is --the result of efficiency neutral 
events or conditions...it can result from changes in 
events, procedures, and processes in weapon system 
procurement. [Ref. 36] 
Cost variances can be either cost growth or cost reduction. 
The SAR estimates cost variance by appropriation and category 
(Ref. 10]. The SAR categories, in order of application, are 


depicted in Table xX. 


TABLE X 
SAR COST VARIANCE CATEGORIES 


— = a 
| CATEGORY DESCRIPTION 
| Economic Change due to economic price 
| level changes. 
| Quantity Change in the number of end 
| units of equipment. : 
| Schedule Change in procurement or 
delivery schedule. | 
| Engineering Alteration to functional system 
characteristics after their 
establishment. | 
Estimating Change due to correction of a 
baseline cost estimate error or 
assumption. 
| 
| 
| 




















Change due to natural disaster, 
work stoppage, and other 
unforeseen events. 












| 


| 


| 


Change for training and support 
equipment, data, and initial 
spares. 


Support 
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This thesis disregards the cost variance in the Economic 
and Quantity categories and emphasizes that associated with 
the Schedule, Engineering, Estimating, Support, and Other 
categories. Justification for the exclusion of the first two 
categories is as follows: 

@ Inflation is notoriously hard to predict. While its 
existence and effect on program cost is known (inflation 
causes cost growth), the magnitude of the cost growth is 
uncertain. 

@® Quantity changes are unpredictable. Over the average 
program’s ten year path to production, both the magnitude 
and effect of those changes are uncertain. 

@® Program management decisions do not influence inflation 
and quantity changes per se. The program manager simply 
accepts the DOD mandated policy and works with it. 

The SAR-based sensitivity analysis attempts to improve the 
original cost estimate’s accuracy. Subsequent examples show 
analyses that remove the unpredictable effects of inflation 
and quantity, and estimate the cost variances associated with 
the Schedule, Engineering, Estimating, Support, and Other 
categories. This particular approach, as subsequent examples 
show, can lead to more accurate cost variances and more 
accurate cost estimates. 

Since the dawn of the republic, analysts and managers have 
consistently underestimated the cost of future weapon systems. 
The earliest documented case involved congressional attempts 
to provide the Navy with six frigates in 1794 [Ref. 36]. The 
subsequent buy was plagued with schedule delays, cost growth 


and mismanagement, due primarily to the political requirement 
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to build the vessels in six different shipyards in six 
different states. In the subsequent 200 years, very little has 
changed. As an Air Force sponsored RAND report that normalized 
data for inflation and quantity effects states: 
...AS an estimating goal, we might hope that, on 
average, our cost estimates are unbiased with a mean 
cost growth of zero, and that accuracy improves over 
time as a function of improved information...our 
results indicate that cost estimates are in fact 
systematically biased toward underestimation: cost 
growth is about 20% at both the planning (Milestone I) 
and development (Milestone II) baselines, falling to 
about 2% at the production (Milestone IIIa) 
baseline...there is a very high variance around those 
averages...the data is highly skewed towards cost 
growth...and distribution does not improve 
significantly over time as better quality information 
becomes available. [Ref. 28] 
Cost growth seems to be the program norm due to underestimated 
costs in the five categories mentioned above. This point is 
further reinforced by Bliss of ASD(PA&E), who discovered that 
of the cost growth measured in the above categories, more than 
66% results from discretionary decisions made by program 
management subsequent to full-scale development [Ref. 25]. 
The ultimate goal of the sensitivity analysis is accurate 
cost estimates that allow objective comparisons of competing 
P-3 COEA alternatives. The thesis disregards the cost 
variances associated with inflation and quantity changes due 
to their inherent unpredictability. In the COEA, all cost 
comparisons are made in base year (FY 1993) dollars, and each 
COEA alternative is evaluated holding system quantities 


constant. This minimizes the above effects. 
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The methodology proposed uses data extracted from SAR 
documents and the RAND Defense Systems Cost Performance Data 
Base (DSCPD), currently under development for ASD(PA&E) [Ref. 
9]. First, the methodology for obtaining both data sets is 
described, and differences between the author’s and the RAND 
approach are noted. Second, the data are presented in tabular 
form. Finally, the author’s and RAND values are compared and 


predicted historical cost variance for the COEFA is determined. 


C. METHODOLOGY 
The methodology for obtaining the predicted historical 
cost variance is as follows: 
® Tailor the SAR data to fit acquisition program needs, 
@ Determine the program baseline, 
@ Normalize the data for inflation effects, 
@ Normalize the data for quantity effects, and 
@ Use a weighted average method to aggregate programs. 
Specific problems encountered and techniques used in the five 
major methodology areas are discussed below. 
1. Tailoring the Data Base 
A list of 225 current and former programs that 
required SARs was provided by the Office of the Undersecretary 
of Defense (Acquisition) /Acquisition Policy and Program 
Integration/Program Management (OUSD(A/AP&PI/PM)) [Ref. 37]. 
From the list, 15 programs were selected that were most like 


the COEA alternatives. The subset evaluated included all 
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large, non-fighter/attack aircraft, both new production and 
modified. The only eligible program not included was the C-5A, 
which never reported in base year dollars, thereby excluding 
its use. Drezner, of RAND, found 15 other early programs in 
the SAR data having this problem [Ref. 28). 

The diverse Bomber/Transport/Non-Strike aircraft 
subset is detailed in Table XII. The subset includes ten Air 
Force and five Navy aircraft. Of the 15 aircraft, eight are 
new production aircraft and seven are aircraft modification 
programs. Program SAR reporting start years range from 1968 to 
1991, with seven programs reporting before 1980 and eight 
after. Although a small data set, this subset of programs is 
representative for the COBA. 

2. Determining the Baseline 

There can be three different baseline estimates (BE) 
for each program; the Planning Estimate (PE), the Development 
Estimate (DE), and the Production Estimate (PdE), with cost 
variance measured from each. Drezner, of RAND, measured cost 
variance from each baseline separately by subtracting the BE 
from the current estimate (CE). This is done because 
aggregating programs with different baselines blurs 
fundamental distinctions relating to program maturity and 
information availability [Ref. 28]. 

The author’s approach is different in that programs 


with different baselines are combined. Four of the eleven 











programs in Table XII have production baselines versus 


development baselines. It should be noted that’ these 
particular programs never had development baseiines, only 
production baselines; they entered the acquisition process at 
a sufficient level of maturity (Milestone IIIa) to omit the 
development phase entirely. Since the production baselines in 
question include all the associated program cost variance, and 
because they are modification programs representative of one 
of the three COEA alternatives evaluated, they are included to 
preclude the already nominal data base from growing smaller. 
3. Normalizing for Inflation Effects 

The first correction to the measured cost variance 
involves normalizing for inflation effects. This is 
accomplished by making all calculations in base year dollars. 

4. Normalizing for Quantity Effects 

The second correction required involves normalizing 
the data for quantity effects. The author’s approach, 
described in Hough [Ref. 26], involves adjusting procurement 
costs by the SAR "Quantity" (Q) variance category amount. The 
correction, as depicted in Table XI, should capture 
approximately 75% of all quantity effects [Ref. 38]. The 
remaining 25% of effects are not captured because not all cost 
variances due to quantity changes are reported under the 
"quantity" category. According to DODI 5000.2-M: 


All quantity changes will be calculated using the 
baseline cost-quantity relationship in effect (PE, DE, 
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or PdE)... The difference between the cost of the 
quantity change based on the baseline cost-quantity 
relationship and the cost based on the CE cost- 
quantity relationship will be assigned to schedule, 
engineering, and estimating categories, as 
appropriate. [Ref. 10]. 


The primary reason to recommend the author’s approach is speed 
and ease of use. At this early point in program life, it is 
accurate enough for COEA purposes. 


TABLE XI 
DATA ADJUSTMENT FOR QUANTITY VARIANCE 


| THe of Quantity Change waitkey chasse: ||| cNomuiteding to Baseline 
! plained 









With the arrival of the annually updated RAND DSCPD, 
another potential source of cost variance information is 
available to cost growth analysts [Ref. 9]. One primary 
difference between the DSCPD cost variance information and 
that previously described is the quantity normalization 
method. RAND uses a method that captures more of the quantity 
effect. According to Hough: 


...The method first requires the determination of all 
reported quantity variance (that is, the dollar amount 
reported under the "quantity" variance category, as 
well as all dollar amounts reported in the other 
variance categories but identified in the [SAR] 
Marrative as quantity-related. A net procurement 
variance is then calculated y subtracting the 
quantity-related variance from the total procurement 
cost variance. The net procurement variance is then 
normalized to either the baseline or currently 
approved quantity using a cost-quantity curve. 
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Normalization of the net procurement variance assumes 
that this residual, which is not explicitly attributed 
to quantity change in the SAR, is, nevertheless, 
aaa | influenced by a change in quantity. [Ref. 

For COEA purposes, if working with the raw data 
contained in the SAR, the author’s method is preferred. If the 
RAND DSCPD is available, its use is encouraged because more 
accurate results are obtained. This is documented in Hough 
{Ref. 26], who conducted a comparison of the author’s and 
RAND’s techniques. 

5. Weighted Average Aggregation 

Total subset cost growth, after all individual program 
cost growths are calculated, is derived by calculating a cost 
growth average weighted by program dollar size. This is 


important, because, as noted by Bliss in Chapter II, program 


size and cost growth are inversely related [Ref. 25]. 


D. TABULAR DATA 

Table XII depicts the data collected from the SARs 
provided by OUSD(A/AP&PI/PM) and found in the RAND DSCPD. Two 
weighted average cost growth factors are delineated. The 
first, using the RAND DSCPD, is 11% (see WT AVG CGR/R). The 


second, from the author’s work, is 6% (see WT AVG CGR/T). 


E. THE COEA HISTORICAL COST VARIANCE 
Before determining which cost growth factor to use, it is 


worth noting that the overall cost growth for programs of this 
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type is much less than the average 33% reported by Bliss [Ref. 
25) and the average 20% measured from the DE reported by 
Drezner [Ref. 28]. This can be attributed to a high level of 
Maturity and low overall risk for this type of program 
compared to the "average" program. 


TABLE XII 
SAR COST VARIANCE DATA 


ee Ei e BP ere Pee 
[Type/Svc/Mod_ [Year [line |Chng |RAND eo CGR/R |Thes CGR/Th 
p-ua/ar seal] pe | - [1.17] 37236| 0.25 |1.17| 37205] 0.24 | 
p-ap/ar/m | 1905| pe | o | 1.00] 34279| 0.20 | 1.00] 36313] 0.20 | 
| 0.01 |0.97| 2597] 0.02 
.76| 10738| 0.05 | 0.76] 13730 
jises | pe .34|_32679| 0.25 [1.23] 3454a| 
[1972 | pe | Sp 2 1.47| 6984! 
jis74 | pe 0.10 
| 6s6| 0.00 | 2.02] 


|1973 | pe | | 653 
[e-4/ar [1976 | pe | 0.02 
| 2438] 0.02 |1.05| 2463) 0.01 
.o1| 6499] 
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designs and modifications of previous aircraft. They used 
relatively non-exotic technology and materials, significantly 
reducing program uncertainty. Finally, all three sets of data 


were normalized for inflation and quantity effects. 
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The COEA historical cost variance used is the RAND figure 
of 11%. The RAND method, since it captures more of the 
quantity effects as opposed to the author’s, provides larger 
quantity adjustments and larger cost-growth factors when 
quantities are decreasing. Since it is more representative of 
actual past program performance, it is the cost growth factor 


of choice. 


F. SUMMARY 

After a cost estimate is made, some effort should be made 
to improve estimate accuracy. Historically, program cost 
estimates have been less than actual program costs, leading to 
cost growth. An author-developed five step method for 
determining historical cost growth is as follows: 

@® Tailor the data base for the particular COKA programs, 
@ Determine the program baseline, 

@® Normalize the data for inflation, 

@ Normalize the data for quantity, and 

® Use weighted average program aggregation. 

After performing the five step method for this thesis, two 
COEA historical cost growth variances were calculated. Because 
the RAND method is more representative of past program 
performance, its estimate of 11% cost growth is used in this 
thesis. The cost variance does not take into account the 


effects of inflation and changes in quantity. 
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V. OPTIMIZATION MODEL DESCRIPTION 


In this chapter, MPAMOD1, the analytical tool for 
implementing the P-3 COEA alternatives, and its assumptions 
are described briefly. Next, a list of differences between 
MPAMOD1 and MPAMOD and a description of required model input 


and output data are presented. 


A. DESCRIPTION 

MPAMOD1 minimizes life cycle cost (LCC) over a long-term 
planning horizon for each of the three COEA alternatives while 
determining schedules for modification or new aircraft 
procurement. Inventory flow balance constraints allow for any 
combination of new production, avionics modification, service 
life extension via SRP and SLEP, aging and aircraft 
retirement, depending on the scenario under consideration. The 
planning horizon is defined as 1993 to 20 years after 
alternative initial operational capability (I0C)>. Cost 
minimization is restricted, however, since the model is 
subject to the following budget, physical, and effectiveness 
constraints: 


® Annual budget limits, 


5IoCc is defined as the year the eighth aircraft modification 


or eighth new production aircraft enters the fleet inventory. 
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@ Annual procurement quantities for new aircraft, avionics 
modifications, and SRP and SLEP service life extension 
programs, 

@ Minimum and maximum inventory levels by year, 

® Minimum MOE by year and mission area, and 

@® Minimum average P-3 fleet life remaining goals. 

Some constraints, such as annual budget limits, procurement 
quantities, and inventory levels, can be very rigid or very 
loose. For example, the annual new aircraft procurement rate 
can be specified, or minimum and maximum procurement rates can 
be specified and the model will determine the number of new 
aircraft to procure. 

The 251 aircraft that comprise the existing P-3 inventory 
are divided into 95 cohort groups of aircraft with similar 
characteristics to restrict model size and solution time. For 
each year of the planning horizon the model decides whether 
these existing aircraft remain in inventory unmodified, are 
modified, or are retired. 

Each COEA scenario is formulated as a mixed-integer 
network flow model with side constraints where aircraft are 
moved over time in yearly increments [Ref. 32]. MPAMOD1 is 
executed in the General Algebraic Modeling System (GAMS) [Ref. 
39]. Although GAMS can be interfaced with many different 
commercially available solvers, the one selected for use is 
the X-System solver [Ref. 40]. This solver has handled similar 
models in the past, such as the PHOENIX, Drash, and Osborn 


models. MPAMOD1 differs from Osborn’s MPAMOD as follows: 
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@ MPAMOD1 revises MPAMOD’s cohort groups to allow for 
differentiation between active and reserve P-3C aircraft 
and P-3C Update II and Update III versions. 

@ MPAMOD1 incorporates the P-3 annual Phased Depot 
Maintenance (PDM) program instead of the Standard Depot 
Level Maintenance (SDLM) program. 

@® Annual Operations and Maintenance (O&M) and Phased Depot 
Maintenance (PDM) costs are calculated for the program’s 
out years. 

@ MPAMOD1 considers aircraft age and life extension 
modification programs when calculating O&M costs. Newer 
aircraft and aircraft receiving SRP and SLEP modifications 
accrue lower O&M costs. 

B. IMPLEMENTATION 
1. Input Data Required 

A typical model scenario covers a planning horizon 
consisting of the first year of program preprocurement cost to 
20 years after IOC. Any mix of new production aircraft, 
avionics modification, and SRP or SLEP service life extension 
programs can be active during the model run. 

Other data required for model execution include annual 
budget limits for each active program and program annual 
production rates. Total P-3 fleet aircraft inventory goals and 
minimum average fleet life remaining goals are also set. 
Aircraft MOEs are obtained from the NAVAIRSYSCOM Warfare 
Analysis Division, NAIR-526 [Ref. 8], and Chapter III cost 
estimates provide unit and preproduction cost data for each 


program. Annual Operations and Maintenance (O&M) and Phased 


Depot Maintenance (PDM) costs are calculated using information 


53 











supplied by the NAVAIRSYSCOM P-3 APML, NAIR-41032 [Ref. 41]. 
Aircraft loss through attrition is not considered in the 
model. MPA attrition historically is very low compared to 
other naval aircraft communities, and its exclusion should not 
affect MPAMOD1 results. Detailed descriptions of model 
required input data follow. 
a. Cohort Groups 

Aircraft are divided into cohort groups using 
October 1992 structural appraisal of fatigue effects (SAFE) 
data supplied by the NAVAIRSYSCOM Structures Branch, NAIR-5302 
(Ref. 42]. Groupings are based on aircraft age, fatigue life 
remaining, current avionics fit (either P-3C Update II or 
Update III), and active or reserve fleet status. 

The retirement ages for each individual aircraft 
are set in the cohort groups based on fatigue rate (annual 
rate of fatigue life use) and service life extension 
modification program status. An unmodified P-3 is retired at 
30 years; the SRP modification adds 8 years to service life 
and the SLEP modification adds an additional 10 years. 

b. Annual Budget Limits and Production Rates 

Annual budget limits are specified in constant FY 
1993 dollars. Using unit and preproduction (P) costs generated 
in Chapter III cost estimates, budgets are calculated that 
allowed the model to purchase the required quantity streams of 


modifications and new aircraft (Table XIII). 
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The AIP annual budget allowed for one time preproduction 


costs of $35 million and unit costs of $10.5 million, 
calculated from data contained in Table IV, Chapter III 
(Calculated as follows in millions: Program Acquisition Cost 
of $749.6 - NRE of $35 = $714.6; $714.6/68 units = $10.5 
million). There is no production lag time; kits are bought and 
installed in the same fiscal year. 


TABLE XIII 
BUDGET QUANTITY STREAMS 
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For the new aircraft, ORION II, the production line 
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opens in FY 1996. The budget permits three years of 
preproduction costs that are calculated from ORION II NRE and 
RDT&E costs (in millions: FY 1993-$163, FY 1994-$228, FY 1995- 
$132). Subtracting these costs from program acquisition costs 
found in Table VI, Chapter III, yields a budgeted unit cost of 
$96 million. A three year production lag is built in. For 
example, aircraft purchased in FY 1996 reach the fleet in FY 


1999. 
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The SRP budget line is calculated as follows: 
first, preproduction costs of $6.8 million are budgeted in FY 
1994; second, unit costs of $7.3 million are calculated from 
data contained in Table VIII, Chapter III, by subtracting NRE 
costs from program acquisition costs and dividing by 193 kits. 
There is a two year production lag built in; aircraft SRP 
modification kits purchased in FY 1994 are installed in FY 
1996. 

The SLEP program’s budget accounts for $90 million 
of preproduction costs in FY 1998. Unit costs of $3.5 million 
were supplied by the P-3 Program Manager, PMA-290. There is no 
production lag time involved with the SLEP program. 

c. Inventory Goals 

Navy P-3C fleet inventory goals are set at a 
minimum of 251 aircraft and a maximum of 275 aircraft. The 251 
aircraft lower limit represents the number of P-3C aircraft 
currently in the fleet. 

d. P-3C Average Fleet Life Remaining Goals 

Average fleet life remaining goals must be set so 
that the model modernizes the fleet through purchase of new 
aircraft and/or incorporation of SRP and SLEP life extension 
programs. Average fleet life goals are set at 22 years in 
1993, and decline to 15 years of average fleet life remaining 


in year 2012. 
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e. Measures of Effectiveness 

MOEs are obtained from the NAVAIRSYSCOM Warfare 
Analysis Division, NAIR-526 [Ref. 8]. The AIP modified 
aircraft and the new production ORION II aircraft are three 
times more effective at performing the ASUW mission than a 
baseline P-3C, based on detection ranges and amount of area 
covered. The MOEs are entered on a scale from 0 to 1 where 1 
is the best. Therefore, a baseline P-3C has an ASUW MOE of 
0.3, an AIP modified P-3C has an ASUW MOE of 0.9, and a new 
production ORION II has an ASUW MOE of 1.0. 

f£. O&M and PDM Budget and Costs 

The O&M and PDM budget is calculated using data 
supplied by the NAVAIRSYSCOM P-3 APML, NAIR-41032 [Ref. 40]. 
O&M costs are $1,909 per flight hour (FH), and are detailed as 
follows (FY 1993 dollars): Fuel and Oil - $728/FH, Maintenance 
Man Hours - $306/FH, Aviation Depot Level Repair Costs - 
$550/FH, and Consumables - $325/FH. Annual O&M costs are $1.26 
million per aircraft for a baseline P-3C with between 16 and 
25 years of service life. PDM costs are $151,000 per aircraft 
per year. This provides sufficient funds to send one quarter 
of the aircraft through the PDM program every year. 

O&M costs are calculated using aircraft age and 
status as inputs. An older aircraft costs more to maintain 


than a newer aircraft. Aircraft that have been modified by the 




















SRP or SLEP programs also cost less to maintain than 
comparable unmodified aircraft. 
2. Output Data Generated 
MPAMOD1's objective is to minimize a scenario’s total 
LCCs through optimum implementation of modernization programs 
and/or procurement over the model’s planning horizon. The 
model also provides an aircraft modification induction 
schedule and shows where small increases of funding over 
budgeted amounts in certain years can result in significant 
downstream operational gains. This data is significant for 
program managers because it can support plans calling for 
increased program resources. The model generates various 
reports to allow analysis of the subsequent output, including: 
@® Annual aircraft inventory level by program type, 
® Annual total costs by program type, 
@® Annual average fleet life remaining, 
@® Annual average ASUW fleet effectiveness ratings, 
® Projected out year inventory levels and costs, and 
@® Annual program specific budgets. 
The output data for the nine specific scenarios are now 


examined. The following chapter details the results. 
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VI. COEA RESULTS 


In this chapter, a detailed summary and discussion of the 
model’s output data are presented by scenario. Cost 
effectiveness of modified aircraft versus new production 
aircraft are compared, assuming that the modified aircraft has 


a shorter service life. 


A. DETAILED DATA SUMMARY 

Nine separate scenarios are run to evaluate the three COEA 
alternatives and determine the service life extension 
program’s effects. The scenarios are detailed in Table XIV. 


TABLE XIV 
MPAMODi COEA SCENARIO DESCRIPTIONS 








[ SCENARIO DESCRIPTION, = 


status Quo (SQ) 
SQ + ASUW Improvement Program (AIP) 


SQ + AIP + (AIP x 11% Cost Growth) 
SQ + New Aircraft (ORION II) 
SQ + ORION IT + (ORION II x 11% Cost Growth) 


r 
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hr 










SQ + Sustained Readiness Program (SRP) 


SQ + AIP + SRP 


SQ + SRP + Service Life Extension Program (SLEP) 


SQ + AIP + Baie + ee 








The scenarios are consistent with the general procedures 


for COEAs found in DODI 5000.2-M [Ref. 10]. They are the 
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feasible scenarios envisioned for the P-3 community during the 
20 year planning horizon of the model, given the budgetary 
considerations previously detailed in Chapter I. One of the 
alternatives, Scenario 1, represents the status quo. Addition 
of the AIP (Scenario 2) results in an improved version of the 
current program. Scenario 4 is a "separate alternative" (the 
new production aircraft alternative) whose costs and benefits. 
can be measured against the previous two scenarios to 
determine if significant advantages are gained through its 
implementation. The other scenarios allow for service life 
extension program and historical cost variance effects to be 
examined. 

The thesis scope limits the options that can be examined 
to those listed above. For the COEA to be truly effective, 
other air options previously mentioned, such as UAVs and 
satellite based systems, should be investigated. 

The data from the nine COEA scenarios is summarized in 


Table XV, with subsequent explanations of data categories. 
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TABLE XV 


COEA SCENARIO DATA 
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The data categories for Table XV are as follows: 


# OF A/C - The number of aircraft remaining in the P-3 
fleet at the 10 and 20 year marks, respectively. 


LIFE REM - The average life remaining per aircraft at the 
10 and 20 year marks, respectively. 


ASUW MOES - The average ASUW MOE per aircraft at the 10 
and 20 year marks, respectively. It is measured on a scale 
from 0.1 to 1.0, where 1.0 is the best. 


ASUW MOE/20 YRS - The total ASUW MOE is calculated as 
follows: average ASUW MOE at 20 years x number of aircraft 
remaining at 20 years. It is a measure of total aircraft 
inventory and favors scenarios that have increased 
aircraft totals. 


ANN $$ - Annual LCCs in millions of FY93 dollars summed 
over the model’s 20 year planning horizon. 


OUT YR $$ - O&M and PDM costs in millions of FY93 dollars 
in the years between the end of the model’s 20 year 
planning horizon and IOC. 


TOTAL $$ - The sum of annual and out year costs. 


TOT A/C YRS - The total number of aircraft inventory years 
calculated by adding the number of scenario aircraft 
remaining in the inventory each year. 


A/C FLT HRS/YR - The number of flight hours each aircraft 
flies in one year. 


TOT INV FL HRS (MIL)- The total projected fleet flight 
hours flown in the particular scenario, calculated by 
multiplying the total number of aircraft inventory years 
by 660 flight hours per year. 


AVG LCC/FH ($000) - The average LCCs of the entire P-3 
fleet in thousands of FY93 dollars per flight hour flown. 
The scenario time horizon extends from 1993 to 20 years 
after program IOC. It is calculated by dividing total 
costs by total inventory flight hours. 


MOE/LCC RAT x 100 - The ratio of total ASUW MOE at 20 
years to AVG LCC/FH multiplied by 100. It is an indication 
of cost versus effectiveness. 


COST RAT - The ratio of AVG LCC/FH to the status quo’s 
(Scenario 1) AVG LCC/FH minus 1, expressed as a percent. 
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It is an indication of the increase in cost over the 
status quo. 


@ AVG 20 YR MOE RAT - The ratio of average MOE at the 20 
year mark to the status quo’s, minus 1, expressed as a 
percent. It is an indication of tthe increase in 
effectiveness over the status quo. 

@ NET TOT MOE - The scenario’s total ASUW MOE at 20 years 
minus the status quo’s total MOE. It is a measure of both 
remaining inventory levels and the ASUW effectiveness of 
those inventories. 

B. DISCUSSION OF OUTPUT DATA 

The output data is discussed using data from Table XV and 
information gleaned from an in-depth study of MPAMOD1 output. 
First, experience gained from using the model to perform this 
COEA analysis is discussed. Second, the status quo is examined 
to determine a baseline to measure subsequent scenarios 
against. Third, an incremental improvement to the status quo, 
the AIP, is explored. Fourth, the costs and benefits of the 
new production ORION II are measured against the two previous 
alternatives. Finally, the effects of cost growth are 
examined, and SRP and SLEP life extension program effects are 
documented. 

The COEA alternatives are compared on an equal basis. Both 
the AIP and ORION II alternative place 68 improved ASUW 
systems in the P-3C fleet. 

1. The Model 


The model performed all calculations quickly and 


found good solutions for the given COEA scenarios. The model 
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is under-utilized in scenarios where top line budget 
constraints and annual procurement quantities are fixed (since 
variables are fixed, the model functions more as a large 
calculator because appropriate trade off decisions are not 
required) . Although it provides the optimal solution given the 
constraints, its real power is evident in scenarios where it 
must choose procurement quantities, such as in cost growth 
Scenarios 3 and 5. In these scenarios it is forced to make 
trade offs between exceeding the allowable budget, long term 
effectiveness, and inventory gains. 

A more effective approach that would utilize the 
model’s capabilities to the fullest in the other scenarios 
would be to only specify annual budgets and programs. The 
model then could choose the best quantities subject to the 
given constraints. 

The model’s detailed output, listed previously in 
Section B.2., provides a wealth of information for program 
out-year planning. This information could be invaluable to the 
program manager if used as a long-term planning blueprint for 
the particular aviation community, regardless of input data 
complexity. 

2. The Status Quo, Scenario 1 

Since model input data in this scenario is restricted 

to current P-3 fleet assets, MPAMOD1 sums O&M and PDM costs in 


Scenario 1 over a 20 year planning horizon extending from 1993 
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to 2012. A schedule is provided that retires approximately 10 
aircraft per year, leaving 62 in the fleet by year 2012. 
Average LCC per flight hour is $2,156, and the ASUW MOE is .25 
in year 2012. 

The status quo is an unacceptable COEA solution. In 
Scenario 1, a less than graceful attrition of current assets 
occurs with no replacement. Life cycle costs are relatively 
high and average ASUW MOE degrades significantly from 0.30 to 
0.25 over the planning horizon. 

3. The ASUW Improvement Program, Scenario 2 

Assuming that the COEA solution must come from the 
alternatives examined, the AIP (Scenario 2) is analyzed first. 
For a 14% increase in average LCCs per flight hour, an 80% 
increase is realized in average ASUW MOE at the 20 year mark 
of the model planning horizon. This seems like a good trade 
off until net total MOE is examined. It is low compared to 
other scenarios because there are only 62 P-3s in the fleet in 
the year 2012. 

This solution still suffers from the degradation of P- 
3 fleet assets due to forced retirements. The large increase 
in average ASUW MOE for a modest increase in average LCC per 
flight hour looks promising, if ways can be found to solve the 


lack of inventory problem. 


65 











4. The New Production Aircraft Program, Scenario 4 

The new aircraft program, Scenario 4, looks less than 
promising. The average 20 year ASUW MOE has increased 136%, 
but average LCCs per flight hour have increased 91%. When 
compared to the AIP program, the ratios are even more 
revealing. The ORION II’s cost ratio is 650% greater than the 
AIP’s, while average 20 year MOE ratio only increases 70%. 

The ORION II is most dominant in the area of net total 
MOE. Its fourfold advantage over the AIP is based on the 
existence of 68 more P-3 aircraft in the inventory in 2012. 

5. Cost Growth Effects, Scenarios 3 and 5 

Two scenarios are run to examine the effects of the 
11% cost growth factor determined in Chapter IV. In Scenario 
3, the cost growth factor is applied to the AIP, resulting in 
a new unit cost of $11.8 million. This is compared to Scenario 
2, the same program without the cost growth. In Scenario 5, 
the cost growth factor is applied to ORION II, resulting ina 
new unit cost of $107.5 million. This scenario is compared to 
Scenario 4 as previously described. 

In Scenario 3, the effects of the 11% cost growth over 
the model’s 20 year planning horizon is hardly noticeable. 
Average ASUW MOE declines slightly at the 10 year mark, and 
total LCCs increase slightly resulting in a modest increase in 


average LCC per flight hour. 
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When examining model output in detail, it is apparent 
how the model accomplished this. MPAMOD1 allowed slight budget 
increases in four of six AIP production years (exceeding total 
budget top line), resulting in procurement of four additional 
units at a net total program cost increase of only $5.8 
million (or 0.10% of scenario total costs). Normally the model 
would be forced to disallow any such top line budget 
violations, but the model constraints are such that if the 
benefits accrued from increased fleet ASUW MOEs outweigh the 
additional costs, the model makes the appropriate trade offs. 

Even with such judicious funding use, the model only 
procured 61 of 68 AIP kits, resulting in the slight ASUW MOE 
decline at the 10 year mark. For either Scenario 2 or 3, the 
average ASUW MOEs are the same at the 20 year mark because the 
model maintains 25 AIP aircraft in the inventory in the year 
2012. It has been forced to retire the other AIP aircraft for 
lack of a suitable life-extension program. 

In Scenario 5, cost growth effects are slightly more 
noticeable. Fleet average ASUW MOEs decline at both the 10 and 
20 year mark, and total MOE declines as well. Total LCCs 
actually decline, but average LCC per flight hour increases 
slightly. The model recommends purchasing 2 more aircraft, 
exceeding proposed procurement budgets in years 5 and 10 by 
0.42% of scenario total costs. The model recommends exceeding 


the budget ceiling early in the procurement profile because 
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accrued benefits exceed costs, and in the last year of the 
profile because the amount required is small. 

In conclusion, cost growth effects of 11% are not 
significant over the model’s planning horizon if model 
recommended procurement profiles are followed. This is 
predicated on obtaining very modest additional program funds. 
The best time to exceed budget limits is early in the 
program’s procurement profile, when accrued long-term benefits 
outweigh additional costs. 

6. SRP and SLEP Programs, Scenarios 6, 7, 8, 9 

The best solution for the ASUW mission need is found 
in the combination of AIP and SRP (Scenario 7) depicted in 
Figure 1. In Figure 1, the MOE/LCC ratio {Total MOE at the 20 
year mark (a measure of both MOE and inventory remaining) 
divided by Average LCC per Flight Hour} is plotted for all 
nine scenarios detailed in Table XIV. The MOE/LCC ratio in 
Scenario 7 increases 281% (from 1.14 to 3.20) over Scenario 2 
(AIP without SRP). This option keeps an inventory of 211 P-3s 
at the 20 year planning mark, causing fleet average ASUW MOE 
to fall slightly due to the spreading of advanced ASUW 
capability over the larger fleet. 

Benefits accrued from adding the SLEP program 
(Scenario 9) are minimal at best, with its greatest effect on 


inventory totals. For a 2% increase in average LCC per flight 


hour, an additional 26 aircraft at the 20 year planning mark 








4 & 
SCENARIO 





Figure 1 MOE/LCC Ratio for all Scenarios 


are realized. MOE/LCC ratio only increases 5%, however, from 
3.20 to 3.36. The majority of the life extension benefits have 
been realized from incorporation oi the SRP. 

The AIP and SRP combination is much better than the 
new aircraft option. Costs over baseline increase only 25% 
(91% for ORION II), and total MOE at the twenty year mark is 


13% greater, primarily due to inventory increases. 


C. COST EFFECTIVENESS COMPARISON 

This section compares the cost effectiveness of a major 
modification program, the P-3C Update IV, to the new 
production ORION II. The service lives for the Update IV 


modified aircraft are not extended, and are significantly 
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shorter than the service lives of the new production aircraft. 
The P-3C Update IV was selected because its unit acquisition 
cost is more than three times greater than the P-3 AIP, 
allowing for a more realistic comparison. Data for the current 
P-3 fleet and the AIP are included to exhibit trends, and are 
presented in Table XVI. 


TABLE XVI 
COST EFFECTIVENESS COMPARISON DATA 


Ca STN CICS 


EE Cana 





Unit costs for the P-3C Update IV were first calculated at 
$31.5 million, and then two cost growth scenarios were 
considered. The first assumed a cost growth of 30%, raising 
unit costs to $41 million. The second assumed a "worst case" 
cost growth of 52%, raising unit costs to $48 million, exactly 
one half the cost of the new aircraft option. The cost ratio 
for this "worst case" scenario is 63%, approximately two 
thirds that of the new aircraft option. This leads to the 
conclusion that, in this particular scenario, the P-3C Update 


Iv modification program is more cost effective than the ORION 
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II. Only when the costs of the modification are completely out 
of control, and experience growth in excess of 50% of the 


original estimate, will the new aircraft option be viable. 


D. FINDINGS 

MPAMOD1 is a powerful analytical tool that handled the 
COEA scenarios very well. It is only limited by the required 
input data and seems more than capable of handling more 
complex scenarios if needed. It provided the necessary 
information to make the proper choice among COEA alternatives 
presented in this thesis. The detailed output it provides is 
very useful as a P-3 community long term planning tool. 

The COEA alternative that provides the best choice for the 
ASUW mission is the P-3 AIP modification combined with the SRP 
life extension program. The COEA status quo, the current P-3 
fleet, is unacceptable because ASUW MOE declines significantly 
over the planning horizon. The new aircraft, the ORION II, 
provides comparable ASUW MOE gains but average life cycle cost 
per flight hour over the planning horizon is 3.5 times 
greater. The SLEP life extension program gains are minimal in 
terms of increased ASUW MOE, although an additional 26 P-3s 
are maintained in an active status at the end of the planning 
horizon through use of the program. The effects of the 11% 
predicted increase in modification and new production aircraft 
costs is hardly noticeable and not a factor if MPAMOD1 


recommended procurement profiles are followed. 
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The P-3C Update IV modification proved more cost effective 
than the ORION II option, except in extreme cases where the 
cost growth for the modification greatly exceeded 50%. New 
aircraft programs were significantly less cost effective than 
the option of modernizing existing platforms, even with the 
commensurate service life gains, primarily due to the high 


program costs. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


As DOD approaches the 21st century, the defense budget in 
general and the weapon system procurement budget in particular 
will experience declines. DOD managers must combine a decision 
making process that uses new analytical tools with accurate 
preliminary information to effectively allocate remaining 
resources for weapon system acquisition. 

The COEA process described in this thesis provides 
justification for this type of acquisition decision. Through 
use of historical cost variances, the process adds quality to 
the analogous cost estimates used as inputs to a model which 
implements a proposed program as effectively as possible. DOD 
Managers can constructively analyze the trade offs needed 
between operational effectiveness and cost, and obtain a 
better priced, more dependable program that meets all stated 
mission needs, while preserving scarce budget resources for 


other programs. 


A. CONCLUSIONS 

The optimization model, MPAMOD1, was successfully and 
effectively integrated into the DOD COEA process. MPAMOD1 
merges estimated program costs, historical cost variances, and 
measures of operational effectiveness for each COEA 


alternative and determines the best implementation plan. 


73 





Output data presented allow valid comparisons to determine the 
best alternative ASUW platform to use in a joint, littoral 
operation. The solution of choice is an ASUW Improvement 
Program (AIP) modified P-3 whose service life is extended 
through the Sustained Readiness Program (SRP) modification. 
Historical cost variance of P-3 cost estimates proves 
inconsequential over the model’s planning horizon. The model 
shows that the cost effectiveness of major modification 
programs becomes doubtful only when modification costs 
experience cost growth exceeding 50% of original cost 
estimates. 

The COEA process is one method to determine optimum DOD 
resource allocation decisions needed to effectively use 
declining defense procurement dollars. The alternative chosen, 
a P-3 modified by both AIP and SRP programs, costs $85.5 
million less per unit to procure than a new production P-3. 
Its estimated life cycle cost per flight hour is $1,416 less, 
and it provides nearly the same ASUW effectiveness. This is a 
productive use of scarce DOD resources. 

Even with incorporation of the SRP life extension program, 
the P-3 fleet reaches the end of its useful life ia 
approximately 20 years. At that time, a new aircraft must be 
procured to preserve the fleet’s MPA capability. 

MPAMOD1 provides sufficient information to make the 
correct COEA decision. The other data that the model provides 


are invaluable as a long term force planning tool. The 
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information is of sufficient detail to enhance the planning 
process, yet still allows decision makers enough flexibility 
in which to implement the recommended solutions. 

The model works best in scenarios where all inputs are not 
preordained. MPAMOD1’s strength is its ability to provide 
solutions to "what if" drills that have many variables and 
complex interactions. Its incorporation into the initial 
stages of program planning is encouraged. 

The value of historical cost variance analysis is not 
apparent in the scenarios examined. Low cost growth 
experienced in small acquisition programs is not significant 
over 20 year planning horizons. Solutions recommended by 
MPAMOD1, such as exceeding procurement budgets by small 
amounts early in a program’s procurement schedule, tended to 
mitigate problems associated with cost growth. 

The effects of life extension programs proves to be very 
beneficial in the scenarios examined. They provide large 
payoffs for a small resource investment. 

The effects of service life on the cost effectiveness of 
major modification programs versus new aircraft procurement 
proves inconsequential as long as modification cost estimates 
remain reasonable. When cost estimates experience growth 
exceeding 50%, the cost effectiveness of the modification 


programs becomes suspect. 
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B. RECOMMENDATIONS 

The COEA process should continue to be used to evaluate 
all proposed DOD acquisition programs, as current DOD policy 
recommends. The information gained through evaluation of 
alternative courses of action proves invaluable for future 
program planning scenarios. The process is an effective method 
of husbanding scarce resources and employing them where they 
will yield the greatest return. 

MPAMOD1 would be more valuable in efforts like these if it 
were easier to use and accessible to more users. To this end, 
the author echoes past recommendations that encourage its 
incorporation into software familiar to a majority of 
potential users. Interfacing MPAMOD1 with a commercially 
available spreadsheet program is one option. 

MPAMOD1 should also be used as far upstree. *.: the 
planning process as possible. This takes advantage of its 
computational power and ability to provide solutions to 
complex planning problems. Using it after decisions have been 
made on primary variables, such as top line budget amounts and 
procurement quantities, needlessly restricts solutions. 

Although cost variance effects prove negligible in this 
case, their incorporation into future COEAs is encouraged. 


With the advent of the RAND Defense System Cost Performance 


Database, such information is readily available for use. 








C. AREAS OF FURTHER STUDY 

Further study is required in the area of data required for 
accurate cost estimates. The most needed items are either 
parametric cost estimating relationships for use in estimating 
modernization and new aircraft programs, or cost analyst 
access to a data base similar to the RAND DSCPD to aid in 
formulation of analogous cost estimates. Cost information is 
guarded jealously and currently hard to find. The author 
recognizes the difficulty of such a recommendation, but such 
items would only improve the cost estimation process. 

A model similar to MPAMOD1 should be designed to evaluate 
the effective use of Naval Aviation resources in all areas. 
Such a model could either examine individual aircraft models 
and variants like P-3s, or whole classes of aircraft such as 
fighter/attack or helicopters. Naval Aviation is currently at 
a crossroads, with many critical decisions to make. A model of 


this type could greatly aid that decision making process. 
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APPENDIX A. DETAILED COST ESTIMATES 


All cost estimates in this appendix are analogous cost 
estimates. Analogous cost estimates start with an existing 
systems actual cost, and linking relationships are then 
developed to estimate the cost of the proposed system. The two 
systems costed should be similar in nature. 

Analogous cost estimates are considered crude and 
unrefined if the original cost data is suspect or the linking 
relationships between the two estimates are not developed 
properly. The more similar the systems are, the better the 
analogous cost estimate. 

The analogous cost estimates in this appendix are not 
based on source cost data, which was unavailable, but on 
information pieced together from a wide variety of sources 
defined in subsequent sections. Although the estimates should 
be accurate because they are based on similar P-3 systems, 
without access to the original source material their quality 
must be suspect. Every effort has been made to make them as 


accurate as possible. 


A. COST ESTIMATE FOR P-3 ASUW IMPROVEMENT PROGRAM 
The P-3 AIP cost estimate is an analogous cost estimate 
that uses historical data obtained from the P-3 Program Office 


(PMA-290), NAIR-524, and the P-3 Avionics Support Program 
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Office (ASPO), rounded to the nearest thousand dollars. Based 
on the historical data, the unit cost of airframe 
modifications and associated hardware ("A" kit cost) is 
$970,000; and the installation cost per unit is $1,185,000. 
The unit cost of equipment ("B" kit cost) for the P-3 AIP cost 
estimate is summarized in Table XVII. 

TABLE XVII 


P-3 ASUW IMPROVEMENT PROGRAM B KIT COST ESTIMATE 
(THOUSANDS OF FY93 DOLLARS) 
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Multiplying these costs by the appropriate annual 
quantities procured in Table XVIII yields the costs entered in 
the appropriate categories. For example, FY 1994 KIT A costs 
are derived by multiplying $970,000 x 13 systems for a total 
cost of $12,610,000. 

The total program acquisition cost, by year, is presented 
in Table XVIII. It is comprised of flyaway cost for quantity 
of units purchased (including NRE), support and equipment and 
spares. Unit costs are also calculated. 

TABLE XVIII 


P-3 ASUW IMPROVEMENT TOTAL PROGRAM COST 
(THOUSANDS OF FY93 DOLLARS) 
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The primary AIP cost estimate unknown is NRE costs. NRE 
costs ranged from a $12 million contractor estimate to a $50 


million government estimate. This estimate’s $35.0 million NRE 
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figure is based on the program’s Statement of Work (SOW), 
which requires minimum software integration for program 
implementation, and the program’s acquisition strategy of 
"streamlining". Costs calculated in Table XVIII are summarized 


in Table IV, Chapter III. 


B. COST ESTIMATE FOR P-3C UPDATE IV 

The P-3C Update IV cost estimate is an analogous cost 
estimate that is presented in thousands of dollars. System 
description and unit flyaway "B" kit cost are presented in 
Table XIX. 

Since a significant portion of Update IV is composed of 
new production systems, the cost estimate contains more 
elements of uncertainty. The cost estimate is based on 
historical data obtained from PMA-290, and, despite 
appearances, the data is judged to be no more accurate than 
the P-3 AIP cost estimate presented previously. 

The "A" kit cost for the P-3C Update IV is $3,779,600. The 


installation cost is $3,237,400. 
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TABLE XIX 
P-3 UPDATE IV UNIT FLYAWAY COST ESTIMATE 
(THOUSANDS OF FY93 DOLLARS) 
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The total program acquisition cost, by year, is presented 


in Table XX. It is composed of flyaway cost for quantity of 
units purchased (including NRE), support and training 


equipment, spares, and RDT&E. A program acquisition unit cost 


is also calculated. 
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TABLE XX 
P-3 UPDATE IV TOTAL PROGRAM COST 
(THOUSANDS OF FY93$) 
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Total costs and unit costs calculated in Table XX are 


displayed in Table VII, Chapter III. 


C. NEW PRODUCTION P-3 COST ESTIMATE 

The ORION II’s analogous cost estimate is based on 
installed systems that are currently flying in fleet P-3s. The 
most uncertainty surrounding the estimate concerns the new 
airframe modifications, especially the fuselage "stretch" and 
the provisions for inflight refueling. The cost estimate is 
based on historical data obtained from PMA-290 and NAIR-524. 


The aircraft unit cost is presented in Table XxXI. 
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TABLE XXI 
ORION II UNIT FLYAWAY COST 
(MILLIONS OF FY93 DOLLARS) 


CATEGORY COST 
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The total program acquisition cost, by year, is presented 
in Table XXII. It is composed of flyaway cost for quantity of 
units purchased (including NRE), support and training 
equipment, spares, and RDT&E. Unit costs are also provided. 


Total costs and unit costs calculated in Table XXII are 


displayed in Table VI, Chapter III. 
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TABLE XXII 
ORION II TOTAL PROGRAM COST 
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APPENDIX B. HISTORICAL COST DATA 


Three completed acquisition programs are sources for P-3 
historical cost data. The completed program data was collected 
from the program’s terminal SAR. In addition, a mythical 
aircraft, the P-11, is created using Hess and Romanoff’s CER 
for all aircraft mission types [Ref. 23]. The P-11 
calculations are presented in the following paragraph. 

The P-11 calculations for 68 aircraft are as follows: 
Aircraft Empty Weight (EW) - 82,000 lb 
Aircraft Maximum Speed (SP) - 410 kn 
Total program cost for 100 aircraft (thousands of $1977): 

= 2.57 (EW*.798) ( SP*.736) 
= $1; 795,317.51 
Total program cost for 68 aircraft (thousands of $1977): 


COST (100) * [(68/100)]*.401 


1,795,317.51 * .86 


$1,538,074.39 
Convert to 1993 dollars: 
= (1,538,074.39/.3835) *1000 
= $4,010,624,222 
Average unit airframe cost: 
= 4,010,624,222/68 = $58,979,768 
Engine costs are $6.64 million and avionics costs are $20 


million. Armament and other costs are $0.2 million. NRE costs, 
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added to the airframe cost category, total $270 million. 
Support equipment and spares cost is 20% and 8% of total 
flyaway cost, respectively. The cost of RDT&E is 15% of 
flyaway, and military construction is negligible. 

The historical cost data is summarized in Table XXIII. It 
also is presented in Table IX, Chapter III. 


TABLE XXIII 
P-3 HISTORICAL COST DATA 
(MILLIONS OF FY93 DOLLARS) 
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Notes: (1) (GSE + TRG EQ/OTH COSTS) /FLYAWAY COSTS 
(2) RDT&E COSTS/FLYAWAY COSTS 
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